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Summary 

In 2002, voters in Florida approved a constitutional amendment limiting class sizes in 
public schools to 1 8 students in the elementary grades, 22 students in middle grades, and 
25 in high school grades. Analyzing statewide achievement data for school districts from 
2004-2006 and for schools in 2007, this study purports to find that “mandated [class-size 
reduction] in Florida had little, if any, effect on cognitive and non-cognitive outcomes.” 
The study has four flaws that, taken together, invalidate it as an evaluation of class-size 
reduction: 

1) The data used are drawn from grades 3 and 4 to 8, where the likelihood of finding 
class size effects is small. 

2) The differences in class sizes of two comparison groups (treated and untreated) 
range from about 0.5 to about 3.0 students, all too small to make a difference educational- 
ly- 

3) School and district average class sizes are used in the analysis rather than the ac- 
tual sizes of classes in which students were enrolled. 

4) The comparison is between two sets of districts, both with small classes, differing 
only in whether state funding was used in a focused or general way. This study actually 
found that administrative discretion in spending state class-size reduction funds did not 
affect students’ academic performance. 




Review 



I. Introduction 

Despite a strong research base that supports 
the effect of class-size reduction (CSR) on 
student achievement, 1 arguments persist 
about the efficacy of reducing class sizes. 

In 2002, voters in Florida approved a consti- 
tutional amendment to limit class sizes in 
public schools to 18 students in the elemen- 
tary grades, 22 students in the middle 
grades, and 25 in high school grades. The 
policy was to be phased in over a period of 
years but implemented fully by 2010-201 1. 2 
The Impact of a Universal Class-Size Reduc- 
tion Policy: Evidence from Florida’s State- 
wide Mandate, by Matthew Chingos, 3 pur- 
ports to evaluate the academic outcomes of 
the policy as implemented through 2007. 

The study consists of two parallel analyses 
of statewide achievement data, one focusing 
on school districts’ average test scores on 
the Florida Comprehensive Assessment Test 
(FCAT) in reading and math for 2004-2006, 
and another using school-level average test 
scores for 2007. The district-level analysis 
was limited to scores in grades 4 through 8, 
and the school-level analysis to grades 3 
through 8. Two groups of students were 
compared in each analysis: those in districts 
or schools with average class sizes smaller 
than the size required by the constitutional 
amendment (the “untreated” group), and 
those in districts or schools with average 
class sizes above the required number (the 
“treated” group). 4 Districts and schools with 
average class sizes that were already smaller 
than the state-mandated class sizes were free 
to use state class-size reduction funds as 
they saw fit. The other schools and districts 
were required to use the funds to reduce 
class sizes. The per-pupil funds provided to 



both sets of districts were the same. 

Two sources of information about average 
class sizes were considered: “official statis- 
tics from the Florida Department of Educa- 
tion (FLDOE)...in core classes in grades 
four to eight (p. 5)” and the study author’s 
calculations from data provided by the State 
Education Data Warehouse (EDW). By and 
large, the two data sets yielded the same 
conclusions. 

II. Findings and Conclusions 
of the Report 

The study addressed three major questions: 

1) Was the constitutional amendment fol- 
lowed by a reduction in average class sizes? 
The Florida Department of Education re- 
ports that “average class size in core classes 
in grades four through eight ... fell from 
24.2 in 2003 to 18.6 in 2009” with class siz- 
es falling slightly more in regular than in 
special education classes (p. 5). 

2) How much did class size vary between 
the treated (those starting with larger than 
mandated class sizes) and untreated (those 
starting with smaller than mandated class 
sizes) groups? In the district-level analysis : 
In Year 1 (2004), the average class size in 
treated districts dropped by 0.9 students in 
grades 4 through 8. In untreated districts, the 
average dropped by 0.1 students. By year 3, 
the average class sizes had dropped by 3.0 
and 1 .4, respectively. That is, untreated dis- 
tricts dropped by 1.6 students fewer than did 
treated districts on average. 5 In the school- 
level analysis : In Year 1 (2007), the average 
class size in treated schools dropped by 2.0 
students. In untreated schools, the average 
dropped by 0.5 students; 1.5 students fewer 
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than the decrease in treated schools on aver- 
age. There was some variation from grade to 
grade and some variation between official 
FLDOE reports and the author’s calculated 
averages that had no noteworthy impact on 
the results. 6 

3) How different was the academic 
achievement of the treated and untreated 
groups? In the district-level analysis, the 
report concludes that “mandated CSR did 
not have a positive effect on student 
achievement above and beyond the effect of 
equivalent additional resources” (p. 21). 
This overall finding of no effect was robust: 
in other words, it did not vary by grade lev- 
el, when Stanford Achievement Test scores 
were used instead of the FCAT scores, or, 
for the most part, with a variety of ap- 
proaches to the statistical analysis. In the 
school-level analysis, the report concludes 
that for both math and reading scores “even 
small effects can be ruled out” (p. 23). In 
general, “The results from both the district- 
and school-level analyses indicate that man- 
dated CSR in Florida had little, in any, effect 
on cognitive and non-cognitive outcomes” 
(Abstract). 

Comparing the treated and untreated groups, 
however, was not the same as comparing the 
achievement of students in small and large 
classes (see Section VI below). 

III. The Report’s Rationale for 
Its Findings and Conclusions 

On the surface the approach taken in this 
study seems logical. All districts in the state, 
and later all schools in the state, were re- 
quired to reduce class sizes. Thus no cross- 
sectional comparison of small classes with 
large classes was readily available. Academ- 
ic performance could have been compared 
before and after class-size reduction took 
effect, examining achievement differences 



in classes, schools, or districts as a function 
of the magnitude of the reduction, but, as the 
report acknowledges, this may have been 
confounded by other changes in school 
funding, curricula, or new programs over the 
same time period. Thus, the author took ad- 
vantage of the fact that some districts had 
class sizes that met the state requirements 
prior to the constitutional mandate. Al- 
though these districts received the same ad- 
ditional funding as other districts, they were 
not required to use it for class-size reduc- 
tion. Funds to these “untreated” districts 
were in essence “block grants” to the dis- 
tricts to be used at their own discretion. Ac- 
cording to the report, “Some surely used it 
to reduce class size, although the class size 
numbers suggest this behavior was modest 
and did not compromise the difference in 
changes in class sizes between the groups” 7 
(p. 9). 

Districts with average class sizes above 
those mandated were required to use the 
funds for class-size reduction (“treated” dis- 
tricts). Approximately 42% of districts were 
classified as treated in grades 4 to 8, and ap- 
proximately 24% of schools. Per-pupil 
spending was the same in treated and un- 
treated districts. In comparison, per-pupil 
spending was increased by 1 1 .7% in treated 
schools and 7.6% in the untreated schools. 
The report states correctly that “school-level 
results have a modestly different interpreta- 
tion than the district-level results” (p. 14), 
since the groups also differed in the addi- 
tional resources available. 

These two groups of districts (and then 
schools) were compared in terms of their 
average class sizes and test scores in reading 
and mathematics with the intention of esti- 
mating the impact of the state CSR initia- 
tive. The assumption is made that students in 
treated districts, and then schools, expe- 
rienced greater class-size reductions and 
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thus were “more affected by the policy” (p. 
6) than were students in untreated districts 
and schools. 

IV. The Report’s Use of 
Research Literature 

The study is presented as an analysis of the 
effects of small classes on student achieve- 

o 

ment, and the literature reviewed pertains to 
class-size reduction. This background re- 
search, however, is not used to inform the 
study and gives an unbalanced picture of 
class size research to date. 

Since 1978 or earlier, it has been found con- 
sistently that benefits of small classes accrue 
mainly in the early grades (K-3) and primar- 
ily are associated with substantial reductions 
in class sizes. 9 These earlier findings are nei- 
ther cited nor incorporated into the study. 
This study began with data from grade 3 
(schools) and grade 4 (districts) without at- 
tention to the earlier grades. The findings 
that multiple years in a small class bring 
greater benefits than do fewer years 10 are not 
mentioned at all. No analyses of class size 
research in the upper grades — the grades 
studied in this evaluation — are cited or de- 
scribed. 11 

The report expresses a bias toward studies 
that conclude class-size reductions have lit- 
tle or no impact on student achievement. On 
page 1 the report asserts that “there is little 
empirical evidence on this question.” Indeed 
there is as much published and unpublished 
research on this topic as on most topics in 
education — much of it showing positive ef- 
fects. The introduction summarizes Alan 
Krueger’s secondary analysis of the Project 
STAR experiment in one sentence, giving 
no attention to primary analyses or to other 
secondary analyses of the data. No research 
is cited regarding other large scale class-size 
initiatives except for the California pro- 



1 2 

gram, ~ which yielded positive effects but 
also created a reduction in teachers’ qualifi- 
cations. The positive effects in California 
were obtained despite the problem of a large 
number of new and emergency-certified 
teachers. lj 

Of greater significance, the report fails to 
consider the strength of conclusions from 
Project STAR (or other evaluations). STAR 
was a large within-school randomized expe- 
riment — the “gold standard” of empirical 
research methods. Instead the report com- 
pares the results with one flawed analysis of 
non-experimental data by another scholar 
that yields negative results. 

V. Review of the Report’s 
Methods 

For the analysis of academic achievement, 
the study used multiple regression analysis 
with student-level data. Several features 
were added to the usual multiple regression 
analysis. In the district-level analysis, dis- 
tricts were weighted by enrollment; dummy 
codes were included for districts so that stu- 
dent achievement was studied relative to the 
achievement of other students in the district; 
standard errors were adjusted for clustering 
(p. 11), although the type of adjustment is 
not stated. The school-level analysis is de- 
scribed as “essentially identical to the dis- 
trict-level analysis” (p. 13). 

The most important methodological feature 
is the use of enrollment and achievement 
trajectories over the years prior to the treat- 
ment. In each analysis, the achievement 
trend for three years prior to the treatment 
was estimated and the treatment was consi- 
dered to have an impact if the change in the 
trend was different for the treated and un- 
treated groups, for example, if the slope of 
the achievement trend became increased for 
the treated districts (or schools) in compari- 
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son with the untreated districts. 14 Three 
years of post-treatment achievement data 
were available for the district analysis 
(2004-2006) and one year of data for the 
school-level analysis (2007). 

The negative findings resulted from the con- 
clusion that the post-treatment trends for 
treated and untreated districts, and for 
treated and untreated schools, were parallel 
(i.e., the slopes are the same). This appears 
to be a correct conclusion based on correct 
analyses. To the author’s credit, the analysis 
also considered a number of alternative 
ways of viewing the data and used robust 
standard errors in tests of significance — all 
producing consistent results. 

VI. Review of the Validity of the 
Findings And Conclusions 

From a statistical standpoint, this study is 
well done. However, it is not an evaluation 
of Florida’s class-size reduction policy and, 
even if it were, it was not based on an ade- 
quate conceptual foundation. There are four 
reasons, in order of increasing importance: 

1) Prior research has shown that benefits of 
small classes are greatest in kindergarten or 
first grade 15 (an argument the report cites on 
page 2) and decrease in magnitude when 
students enter small classes in second and 
third grades, respectively. 16 Consistent with 
developmental theory, in the STAR experi- 
ment students who entered small classes in 
these years had larger and longer-lasting 
academic benefits than did students who en- 
tered in second or third grade. Effects were 
strongest for students who entered small 
classes early and remained in them for sev- 
eral consecutive years. 

If effects are smaller for students who begin 
small classes in second grade than in first, 
smaller in third grade than in second, and so 



on, then there is little likelihood of finding 
effects in higher grades, especially in a study 
that averages all class sizes in a school or a 
district. That is, by studying upper grades, 
the analyses had limited chances of finding 
differences in trajectories because the differ- 
ences may have been small. These certainly 
would not tell the full story implied by the 
report’s title, “The Impact of ... Florida’s 
Statewide Mandate.” 

2) The report finds that class size in treated 
districts dropped by an average of 1.6 stu- 
dents more than in untreated districts. On 
average class sizes in treated schools 
dropped by 1.5 more students than in un- 
treated schools. It’s difficult to imagine that 
instruction would change or that the aca- 
demic or social environment would change 
with the addition or subtraction of 1, 2, or 
even 3 students. Moreover, it would be ex- 
tremely difficult to detect an effect statisti- 
cally based on such a small difference. 18 The 
finding of “little, if any, effect” could have 
been anticipated at the outset from the de- 
sign of the study. 

3) Actual class sizes are hidden when school 
or district averages are reported. This report 
provides information about how average 
class sizes were calculated in this study (in 
the form of two footnotes on page 5), but 
none about variability within schools or dis- 
tricts. In most schools, third and fourth (and 
fifth) grade classes are mostly self- 
contained, but still there is variability from 
class to class. In the years covered by this 
study, individual classes were not con- 
strained to a particular size, and some may 
have been smaller and some larger. Self- 
contained (small) special education classes 
were included in the averages, which sug- 
gests that some regular classes may have 
been quite large. In higher grades, students 
usually move from room to room for differ- 
ent subjects, and may participate in classes 
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of very different sizes. If a small class is 
more efficacious than a large class, the use 
of averages may well obfuscate this fact. 

4) All districts (and then schools) studied in 
this analysis were required by state mandate 
to have small average class sizes. Those 
“treated” received targeted funds to reduce 
average class sizes down to the mandated 
sizes. Those “untreated” had small classes 
prior to the constitutional amendment and 
received a block grant that was not necessar- 
ily used for class-size reduction. Despite 
some speculation — the results “raise the 
possibility that comparison districts were 
able to spend the additional resources more 
productively than the treated districts” (pp. 
21-22) — no infonnation was available about 
how the funds were spent. After the consti- 
tutional amendment was implemented, both 
groups had small average class sizes. 19 

Instead of a study of the impact of class-size 
reduction, this is actually a study of the im- 
pact of providing resources to districts that 
are eannarked for class-size reduction ver- 
sus providing the same amount of resources 
that districts could spend as they wish (an 
accurate paraphrasing of the author’s own 



statement on page 9). These results confirm 
other research showing that providing a 
small amount of administrative budgetary 
discretion has no impact on student 
achievement. They do not show whether 
small classes are beneficial academically 
because the study was not a comparison of 
smaller with larger classes. 

VII. Usefulness of the Report 
for Guidance of Policy 
and Practice 

Despite its title, this report does not address 
the issue of class-size reduction. By being 
presented as an evaluation of Florida’s man- 
dated class size limits, it may lead parents, 
educators, or policy makers to draw faulty 
conclusions about the impact of the pro- 
gram. 

It is possible that Florida’s class-size reduc- 
tion, at least in the upper grades, is having 
little or no impact on student achievement. It 
is also possible that — consistent with the 
beliefs of teachers and parents — small 
classes in the upper grades provide academic 
benefits. The report does not answer this 
question one way or the other. 
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